import { Meta } from '@storybook/addon-docs/blocks'; # Testing Best Practices We need a way to answer hypotheses, empathize with users, and save Engineering time spent iterating. The Digital team strives to be a data-driven, user-centric team that integrates testing & research into our product design and development process. We strive to test early and often throughout the product life cycle, and can work in partnership with outside departments to incorporate testing into our digital products. We support usability testing best practices for implementation of platform iteration and/or innovation. These methods allow for efficient evaluation of design hypotheses and/or identification of opportunities for improvements to inform solutions that will be additive to the user experience (and at the same time, accomplish business goals). Below is a general guideline to how Digital approaches research. Reach out if you have any suggestions or questions you think might be a great to test. ### Testing Guidelines * **If you have a question, think about testing.** Has your question been answered already? Try gathering and reading through Secondary Research to see if the answer is already out there. If the data is old, or industry specific, consider putting together a test plan to better answer your questions. * **Create research tickets*.** Research is time intensive. Understand that research can take various forms of fidelity (eg. Scrappy vs Formal). Try to account for an appropriate amount of time needed to complete your project by breaking things into tasks and partnering with the appropriate teams. If the test is one that requires a large amount of effort to complete, it is extremely important to partner with other teams to garner support and help complete your project. Tickets should include an overview of research goals, questions to be answered, and proposed methodology to help prioritize with other team goals. (*as needed) * **Make a Test Plan.** Document a project’s Objectives / Goals, Questions, Hypotheses, Timelines, Target audiences, Screener Questions (if relevant), and Methodologies in a central place. This will help explain the reasoning behind your research, and set clear boundaries of the scope of your project. ( [Example Test Plan](https://www.google.com) * **Write a Report.** One of the best ways of summarizing and sharing data is by creating a report! Reports are written in a variety of fidelities. A less formal report might include appending a quick list of results to the end of your Test Plan. For more formal reports, creating a more traditional presentation deck to share with stakeholders might be the way to go. The report could list out recommendations, opportunities needing further exploration, or areas of improvement. Make sure to record a link to the test results in our Research Repository. * **Include Stakeholders in your research efforts where appropriate.** Including stakeholders as part of your research process (ex. During planning, conducting, or synthesizing efforts) will broaden your perspectives. Including others also fosters understanding towards potential impacts or discoveries along the way. If stakeholders are not able to take part: add snippets of recordings, or quotes from users to convey pain points in your final report. * **Make Research a part of the beginning, middle, and end of every project.** Make time to make sure research impacts every decision that you make in a project. By testing, we can help the team avoid conflict, spin, and indecision when working together. (ex. Put together a pitch? Run an interview! Finalizing wireframes? Run a card-sort or tree test! Finalizing design direction? Preference test could be your BFF! Want to make sure the product works the way the team intended? Run a usability test! ... and so on...)
## Table of Suggested Tests & Methodologies The list below is not an exhaustive list, but contains a few popular tests and techniques that the Digital team has used in the past to help answer questions. Please add to this list if you'd like to reccomend additional tests.
### Quick Hits Suggestions | Test or Technique | Type | Tools | Effort | Best Time to Run | |--------------------|------|-------|--------|------------------| | **Sentiment Analysis**
Understand user emotions within qualitative feedback, usually a survey. | `Exploratory` | Usability Hub
Hotjar
Google Forms
Survey Monkey | πŸ’š
Low | •-------•
`Beginning`, `End` | | **Preference Test**
Discover user perception of your design by asking users to choose between variations and explain why. | `Evaluative` | Usability Hub
Hotjar
Google Forms
Survey Monkey | πŸ’š
Low | ----•----
`Middle` | | **Secondary Research Gathering**
The summary, collation and/or synthesis of existing research. | `Exploratory` | [Research Repositiory](https://wiki.nypr.digital/pages/viewpage.action?spaceKey=DT&title=User+Testing+and+Research+Repository),
Data Team,
Pew Research Studies,
Edison Data,
Nielson Studies,
The Internet | πŸ’š
Low | •--------
`Beginning` | | **Survey**
A quick and relatively easy way to get data about your users and potential users. | `Exploratory` | Hotjar
Survey Monkey
Google Forms | πŸ’š πŸ”Έ
Low to Medium | •----•---•
`Beginning`, `Middle`, `End` | | **Analytics**
A collection of data that can be used to create hypotheses, containing information of users such as their age, location, interests, or behavior. | `Evaluation` | Google Analytics
Looker
[UX Dashboard](https://datastudio.google.com/u/0/reporting/92ab1235-85f9-47e5-8645-23957d8ad615?s=iTNbwSj8FHQ) | πŸ’š
Low | •-------•
`Beginning`, `End` |
### Exploratory Suggestions | Test or Technique | Type | Tools | Effort | Best Time to Run | |--------------------|------|-------|--------|------------------| | **Personas**
A fictional character created with qualitative data to represent a group of segmented user types, often based on usage of your product. | `Exploratory` | Usertesting.com
User Interviews
Figma
Miro | πŸ”Έ πŸ”Ί
Medium to High | •--------
`Beginning` | | **Archetypes**
Quantitative-based behavioral perspectives of users towards a specific product around a group's needs, motivations and pain-points. | `Exploratory` | Usertesting.com
User Interviews
Figma
Miro | πŸ’š πŸ”Έ
Low to Medium | •--------
`Beginning` | | **Shadowing**
When a researcher accompanies the user and observes how they use the product or service within their natural environment. | `Exploratory` | Usertesting.com
Rainbow Spreadsheet
Trello
Miro | πŸ”Έ
Medium | •--------
`Beginning` | | **Interviews**
A research method involving a researcher asking a user questions with the goal of learning more about that user or the effacy of a product they are using. | `Exploratory`, `Evaluation` | Usertesting.com,
Rainbow Spreadsheet,
Trello,
Zoom,
NYPR Meeting Room or Room Rental | πŸ”Έ πŸ”Ί
Medium to High | •----•---•
`Beginning`, `Middle`, `End` |
### Evaluative Suggestions | Test or Technique | Type | Tools | Effort | Best Time to Run | |--------------------|------|-------|--------|------------------| | **A/B Testing**
A randomized experiment with two or more variants, used to test a hypothesis. | `Evaluation` | Google Optimize | πŸ’š πŸ”Έ
Low to Medium | ----•---•
`Middle`, `End` | | **Card Sorting**
A method used to help design or evaluate the information architecture of a product. | `Evaluation` | Optimal Workshop | πŸ”Έ
Medium | •--------
`Beginning` | | **Usability Testing**
A technique used to evaluate a product by testing with users. | `Evaluation` | User Testing
Guerrilla Interview | πŸ”Έ
Medium | •-------•
`Beginning`, `End` |
### Analysis-Focused Suggestions | Test or Technique | Type | Tools | Effort | Best Time to Run | |--------------------|------|-------|--------|------------------| | **Affinity Mapping**
A tool often used to find patterns, organize data, and generate ideas. | `Analysis` | Trello
Miro
Spreadsheets
| πŸ”Έ πŸ”Ί
Medium to High | •----•---•
`Beginning`, `Middle`, `End` | | **Word Clouds**
A tool used to visually surface reoccuring patterns and trends in qualitative information. Usually paired with a survey. | `Analysis` | Word Repeat Calculator
Word Cloud Generators | πŸ’š
Low | •----•---•
`Beginning`, `Middle`, `End` | | **Rainbow Spreadsheet**
A spreadsheet where researchers can record behavioral patterns presented by research participants during interview sessions. | `Analysis` | [Google Docs Template](https://docs.google.com/spreadsheets/d/1bPg6op9Sk46lFVGaET-fruE0qz-ctNQsxbZKF-5lpn4/edit#gid=0) | πŸ’š πŸ”Έ
Low to Medium | •----•----
`Beginning`, `Middle` |